Several attempts have been made to explore local significant differences in the past years.

As explained in Sec. 4, with Within the first embedding, the optimal value of 0.398 can be reached. As described in Sec. 5, we show the importance of the Mahalanobis kernel using models fit to Fig. S2 compares model predictions for each of these models with the actual test-set outcomes; results Fig. S3 evaluates the predictive log marginal probabilities for the ARD RBF kernel and the Ma-halanobis kernel with posterior sampling across a wide range of training sets with different sizes Mahalanobis kernel is able to learn as the training set is expanded. This can be seen in the optimization results (Figs. 5 and S7) where ALEBO The implied kernel on the embedding is thus stationary. The argument follows that of Prop. 1. Linear embedding HDBO requires selecting a dimensionality for the embedding. The nature of the dimensionality vs. iteration budget trade-off is important in all These same considerations apply to multi-objective optimization.

We thank the reviewers for their detailed reviews and constructive feedback. We will add discussion of this in the extra page. Sensitivity is explored in S9, and ALEBO is shown to be better than prior work. Supplemental: Thanks for the suggestion, we will update to improve clarity! Clarifications: Thanks for pointing these out, we will clarify them.

The optimization of high-dimensional black-box functions is a challenging problem. When a low-dimensional linear embedding structure can be assumed, existing Bayesian optimization (BO) methods often transform the original problem into optimization in a low-dimensional space. They exploit the low-dimensional structure and reduce the computational burden. However, we reveal that this approach could be limited or inefficient in exploring the high-dimensional space mainly due to the biased reconstruction of the high-dimensional queries from the low-dimensional queries. In this paper, we investigate a simple alternative approach: tackling the problem in the original high-dimensional space using the information from the learned low-dimensional structure. We provide a theoretical analysis of the exploration ability. Furthermore, we show that our method is applicable to batch optimization problems with thousands of dimensions without any computational difficulty. We demonstrate the effectiveness of our method on high-dimensional benchmarks and a real-world function.

Bayesian optimization (BO) is a popular approach to optimize expensive-to-evaluate black-box functions. A significant challenge in BO is to scale to high-dimensional parameter spaces while retaining sample efficiency. A solution considered in existing literature is to embed the high-dimensional space in a lower-dimensional manifold, often via a random linear embedding. In this paper, we identify several crucial issues and misconceptions about the use of linear embeddings for BO. We study the properties of linear embeddings from the literature and show that some of the design choices in current approaches adversely impact their performance. We show empirically that properly addressing these issues significantly improves the efficacy of linear embeddings for BO on a range of problems, including learning a gait policy for robot locomotion.